Sensory stimuli delivery systems for head-mounted displays

ABSTRACT

Introduced herein are techniques for bringing scented advertisements into digital content (e.g., visual content and non-visual content). Such technology is able to address a fundamental challenge that is inherent in digital transactions, namely, an inability to accurately gauge different characteristics of a product. Several embodiments pertain to scent delivery systems that are able to produce scent(s) as part of a direct advertising campaign or an indirect advertising campaign. More specifically, a scent delivery system may be configured to produce scents that correspond to features of the digital content. For example, different scented materials can be dispensed based on characteristics of a product that is the subject of a scented advertisement.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/480,223 titled “SCENT MARKETING AND ADVERTISING FOR HEAD-MOUNTEDDISPLAYS” and filed on Mar. 31, 2017, which is incorporated by referenceherein in its entirety.

RELATED FIELD

The present technology relates to the production of sensory stimuli and,more specifically, accessories for electronic devices that producesensory stimuli to increase the realism of content presented by theelectronic devices.

BACKGROUND

Smell is one of our strongest senses. Certain scents often remind us ofmoments (both good and bad) that we first experienced days, weeks, oryears ago. Scents can also make be highly effective in helping usdistinguish items or locations from one another.

Some businesses (e.g., production studios, movie theaters, and amusementparks) have long attempted to enhance interactive experiences byintroducing scents. For example, “Smell-O-Vision” was a system thatreleased scents during the projection of a film so that viewers could“smell” what was happening in the movie. Similarly, the “Smellitizer”was developed by the Walt Disney Company to deliver faint scents on cueat certain locations, such as attractions (e.g., Soarin' OverCalifornia® at Disneyland®) and storefronts (e.g., Main Street Bakery).Scent dispersal systems such as these were often shown to heighten therealism of a fabricated experience, increase sales, and improve customersatisfaction.

Accordingly, entities across different industries have begun using scentas part of a multi-sensory marketing strategy designed to enhance anindividual's experience (e.g., of a location or a particularproduct/service). These enhanced experiences allow customers to developdeeper memories and emotional connections with certain brands, products,services, locations, etc. In fact, some brands have begun using scent todistinguish themselves in the market. Such techniques are generallyperformed as part of a scent marketing campaign.

Scent marketing campaigns can typically be readily introduced bymerchants who occupy a physical storefront. However, scent marketingbecomes increasingly difficult as merchants complete more transactionselectronically and reduce the number of physical storefronts. Saidanother way, the proliferation of electronic transactions has made scentmarketing more difficult because scents must be delivered to prospectivecustomers.

SUMMARY

Smell is a powerful sense that has the ability to evoke differentfeelings and memories, as well as impact a consumer's experience duringa transaction. In fact, smell may influence whether the consumerultimately decides to purchase certain types of products, such ascosmetics, perfumes, colognes, soaps, shampoos, personal hygiene items,etc.

For example, fragrance strips have been placed in magazines to marketsuch products since the 1960s. Studies have shown that fragrance stripsare very effective in grabbing a reader's attention. On average, 64% ofmagazine readers try at least one fragrance strip (or another form ofscent-based advertisement) if fragrance strip(s) are placed in amagazine. Studies have also shown that these scented advertisementsincrease product sales by attracting more attention than unscentedadvertisements.

Introduced here, therefore, are systems and techniques for bringingscented advertisements into other forms of content, including visualcontent (e.g., augmented and virtual reality content) and non-visualcontent (e.g., audiobooks). Such technology can be used to address afundamental challenge that is inherent in digital transactions (i.e., aninability to accurately gauge different characteristics of a product) byengaging a consumer's sense of sight (vision), hearing (audition), taste(gustation), smell (olfaction), and/or touch (somatosensation). Stimulimay also be produced for other sensory modalities such as temperature(thermoception).

More specifically, a stimuli delivery system may produce a sensorystimulus (e.g., a scent) that is related to the content beingexperienced by an individual. For example, the stimuli delivery system(also referred to as a “scent delivery system”) may produce a scent thatis related to a product shown within a virtual or augmented realityenvironment. The product may be the subject of a direct advertisement oran indirect advertisement (e.g., as part of a cinematic film or videogame). The term “product” includes goods and/or services that areavailable for purchase by the individual.

If the individual indicates an interest in the product, the individualmay be asked whether she wishes to purchase the product. This can occurin several different ways. In some embodiments, a notification may bepresented by the computing device used to view the visual content (e.g.,on a head-mounted display (HMD) or a network-connected television). Thenotification may prompt the individual to approve a purchase of theproduct, input financial information (e.g., name, address, and paymentcard details), decline a purchase of the product, or add the product toa queue (often referred to as a “shopping cart”) for further review. Inother embodiments, a notification may be presented by another computingdevice that is communicatively coupled to the computing device used toview the visual content. For example, the individual may receivenotifications on a mobile phone while viewing visual content on an HMDor a network-connected television.

Also introduced here are techniques for generating emotional input basedon characteristics of an individual (referred to as “usercharacteristics”). For example, an emotional input may be generatedbased on the pulse rate or temperature of an individual. These featurescould be measured by an internal sensor that is integrated into thecomputing device (e.g., an HMD) used to view visual content or anexternal sensor that is integrated into some other computing device(e.g., a fitness tracker or Microsoft Kinect®). Emotional input mayaffect the sensory stimuli that are produced by the scent deliverysystem. For example, scents may be produced less frequently or with lessintensity as an individual's heart rate and/or temperature begin torise.

The individual may also have the option of placing the scent deliverysystem in automatic mode or manual mode. In automatic mode, the scentdelivery system decides which scent(s) should be produced and when thosescent(s) should be produced. In manual mode, the individual can eitherturn off the scent functionality entirely or personalize the deliveryparameters (e.g., by specifying a delivery time, a delivery duration, ascent intensity, which type(s) of content should have scentfunctionality, which scent(s) should be produced, which scent(s) shouldnot be produced, etc.).

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention are illustrated by way of exampleand not limitation in the accompanying drawings, in which likereferences indicate similar elements. Various objects, features, andcharacteristics of the present invention will become more apparent tothose skilled in the art from a study of the Detailed Description inconjunction with the accompanying drawings.

FIG. 1A depicts a conventional head-mounted display (HMD) that includesan image display system and an optical display surface configured toreflect light.

FIG. 1B depicts a conventional HMD that includes an image display systemand an optical display surface configured to reflect light.

FIG. 2A is an inside view of a binocular HMD that includes a scentdelivery system.

FIG. 2B is a front view of the binocular HMD that includes a scentdelivery system.

FIG. 3 depicts an example of a network environment that includes an HMDhaving a scent delivery system, a content server, and another computingdevice.

FIG. 4 depicts a scent delivery system that can be affixed to an HMDconfigured to display augmented or virtual reality content.

FIG. 5 depicts a development platform that allows visual content to bespecially designed for presentation by a computing device that isconnected to a scent delivery system.

FIG. 6A depicts an example of a video track that includes a keyframeidentified through image segmentation.

FIG. 6B depicts an example of an audio track that includes a keyframeidentified through audio segmentation.

FIG. 7 depicts an interface through which a user can browse itemsoffered for sale by an electronic commerce platform.

FIG. 8A depicts a digital kitchen environment that is viewable throughan HMD.

FIG. 8B depicts a digital garden environment that is viewable through anHMD.

FIG. 9 includes an overhead map illustrating how a digital environmentmay include one or more scented regions.

FIG. 10A depicts an order interface that may be presented by a computingdevice (e.g., an HMD or a network-connected television) that is used toview visual content.

FIG. 10B depicts an order interface that may be presented by anothercomputing device (e.g., a mobile phone or a wearable device, such as afitness tracker or watch) that is communicatively coupled to a computingdevice used to view visual content.

FIG. 11 depicts a process for generating an instruction set for a scentdelivery system.

FIG. 12 depicts a process for controllably producing scents thatcorrespond with digital content experienced by a user.

FIG. 13 is a block diagram illustration an example of a processingsystem in which at least some operations described herein can beimplemented, consistent with various embodiments.

The figures depict various embodiments for the purpose of illustrationonly. One skilled in the art will readily recognize that alternativeembodiments of the technology described herein may be employed withoutdeparting from the principles of the present invention.

DETAILED DESCRIPTION

Introduced here are computer-implemented techniques and systems forincreasing the effectiveness of marketing, advertising, and brandingcampaigns that target consumers of digital content by controllablyproducing certain sensory stimuli. Stimuli delivery systems (alsoreferred to as “scent delivery systems”) are described herein that canproduce different sensory stimuli based on the digital content beingconsumed at a given point in time. For example, a scent delivery systemmay produce one or more scents based on visual content being shown on ahead-mounted display (HMD). A scent delivery system may be separatelyattachable to a computing device used to view the visual content or maybe integrated within the computing device itself.

Although many of the embodiments described herein involve head-mounteddisplays (HMDs), one skilled in the art will recognize that suchembodiments have been selected for the purpose of illustration only.Such technology could be used in combination with any computing devicethat presents content, including personal computers, tablet computers,personal digital assistants (PDAs), mobile phones, game consoles (e.g.,Sony PlayStation® or Microsoft Xbox®), music players (e.g., Apple iPodTouch®), wearable electronic devices (e.g., a watch or fitness band),network-connected (“smart”) devices (e.g., a television),virtual/augmented reality systems (e.g., Oculus Rift® or MicrosoftHololens®), or other electronic devices.

For example, an individual could wear a network-connected mask (or someother form of headwear) that includes a scent delivery system but allowsthe individual to view visual content shown by another computing device(e.g., a network-connected television). Accordingly, thenetwork-connected mask may be communicatively coupled to one or moreother computing devices. The network-connected mask and the othercomputing device(s) may communicate with one another via a wiredconnection or a wireless connection. In some embodiments, thenetwork-connected mask and the other computing device(s) communicate viaa short-range wireless technology/protocol, such as Wi-Fi, Bluetooth,near-field communication (NFC), cellular radio, infrared,radio-frequency identification (RFID), etc.

A scent delivery system can include one or more reservoirs of scentedmaterial (e.g., liquid or air) that can each be selectively dispensed bya pump at the appropriate time. That is, the scent delivery systemincludes pump(s) for controllably dispensing the scented material(s) incertain quantities/mixtures at certain times. For example, a pine scentcould be produced when the individual is able to view a digital forest,while a vanilla scent could be produced when the individual is able toview a digital kitchen. In some embodiments, the scented material(s) arehoused within capsules that are removable from the scent delivery systemand readily replaceable. Accordingly, the reservoir may be refillableonly when the attachment is separated from the head-mounted device ormodularly replaceable while the scent delivery system is connected tothe head-mounted device.

Unlike traditional scent dispersal systems (e.g., the Smell-O-Vision orthe Smellitizer), the scent delivery systems described herein areportable and allow scents to be delivered directly to a single user ofan HMD. Said another way, traditional scent dispersal systems aredesigned to dispense scents in a single physical location, while thescent delivery systems introduced here can be used regardless of wherethe user decides to experience visual content.

Consequently, scent delivery systems can be used to increase theeffectiveness of marketing, advertising, and branding campaigns thattarget consumers of digital content by controllably producing certainscent(s). Several different use cases illustrate the benefits ofenhancing the realism of digital content (e.g., visual content andnon-visual content) by replacing unrelated real-world stimuli withcontrolled stimuli.

-   -   Direct Advertising: A scent delivery system can produce an        appropriate scent while an individual watches an advertisement        for an item (e.g., a perfume or a deodorant). The individual may        also be permitted to buy the item in real time during the        advertisement. For example, if the individual exhibits an        interest in the scent (e.g., by moving her head closer to the        digital source of the scent), the individual may be prompted to        decide whether she would like to order the item directly through        the advertisement. That is, the individual may be able to        complete a digital transaction without disrupting the current        experience by navigating to a separate channel.    -   Indirect Advertising: Scent(s) related to digital content        experienced by an individual can be intelligently replicated by        a scent delivery system, which allows products already present        in the digital content to be indirectly advertised to consumers.        For example, the scent delivery system may produce a perfume        scent when a female character in a cinematic film approaches the        camera. The individual may be able to order the item in real        time (i.e., as the digital content continues to run) and/or save        a product summary for subsequent ordering. Moreover, a list of        product(s) indirectly advertised throughout the runtime of the        digital content could be shown to the individual following the        conclusion of the digital content. In such instances, the        individual may prompt the scent delivery system to reproduce        scent(s) that were previously produced during the runtime of the        digital content. Indirect advertising techniques are        particularly useful for those forms of content where direct        advertising is undesirable (e.g., due to content flow        disruptions) or difficult to carry out (e.g., due to the        presence of ad blockers).    -   Business Model Improvements: Merchants (i.e.,        manufacturers/sellers of cosmetics, perfumes, food, beverages,        etc.) can work with fragrance companies (i.e., manufacturers of        scented materials) and development companies (e.g.,        manufacturers of scent delivery systems and producers of digital        content) to re-create scents that will improve the effectiveness        of advertising campaigns, the desirability of products, etc. For        example, some or all of these entities may work together to        develop a scented advertisement and generate instructions for        producing the appropriate scent(s) at the appropriate time(s).    -   Promotional Cartridges: Merchants could send promotional        capsules that include scented materials out on a periodic (e.g.,        weekly or monthly) basis. For example, a perfume manufacturer        may distribute promotional capsules each time a new perfume is        developed. As another example, the perfume manufacturer may        distribute promotional capsules as part of the concept testing        process that typically occurs prior to production (e.g.,        multiple capsules could be delivered to potential consumers who        are asked to identify their favorite scent or rate the various        scents).

Accordingly, visual content may be specially designed for presentationby a computing device that is connected to a scent delivery system. Insuch embodiments, a processor can determine whether to dispense ascented material based on instructions that are automatically generatedbased on characteristics of the corresponding visual content or manuallygenerated by an administrator. The administrator may be associated witha merchant, a fragrance company, or a development company. Theadministrator may generate the instructions by working with one or moreof these entities to identify the combination of one or more scentedmaterials needed to accurately reproduce a certain scent. Theinstructions could also be automatically generated based on, forexample, visually identifiable elements in individual frames, audioevents, etc.

The instructions may also specify a delivery timestamp, anintensity/quantity of each scented material, etc. In some embodimentsthe instructions are embedded within the digital content (e.g., encodedin a video track, audio track, and/or metadata), while in otherembodiments the instructions are included in a distinct instruction setthat accompanies the digital content and is separately executable by thescent delivery system. Similar techniques could be used for visualcontent (e.g., applications, games, and cinematic videos) and non-visualcontent (e.g., audiobooks, concerts, and radio programs).

Older, unscented digital content could also be repurposed for use with acomputing device having a scent delivery system. An administrator may beable to tag unscented digital content to create keyframes that cause thescent delivery system to produce scent(s). For example, theadministrator may identify keyframes in a direct advertisement that willbenefit from the production of a scent. As another example, theadministrator may identify keyframes in existing digital content (e.g.,a cinematic film) in order to create an indirect advertisement.

Content modification may be completed using a graphical user interface(GUI) that is supported by a repurposing engine. More specifically, theadministrator could upload unscented digital content to the GUI, whichprompts the repurposing engine to analyze the unscented digital contentto determine whether scent(s) should be produced and, if so, whichscent(s) should be produced. For example, in some embodiments therepurposing engine automatically identifies candidate instances fordispensing scented material(s). The candidate instances typicallyinclude one or more features (e.g., a character, a weather event, or aproduct, such as a perfume bottle or a piece of fruit) that are readilyrecognizable upon performing content analysis. Content analysis mayrequire the repurposing engine execute image processing algorithms,audio processing algorithms, etc. As another example, in someembodiments the repurposing engine automatically determinescharacteristics regarding the production of each scent. Thecharacteristics could include how much scented material should bedispensed (e.g., whether the aroma should be faint, medium, or strong),when the scented material should be dispensed, whether a combination ofmultiple scented materials is necessary to produce a certain scent, etc.The administrator may be able to modify the instances and/orcharacteristics proposed by the repurposing engine. The administratormay also be able to specify additional instances and/or characteristicsusing the GUI.

Disjointed experiences will often occur if scents are produced too earlyor too late. Accordingly, predictive analytics may be employed todetermine when, exactly, a scent should be produced for a givenkeyframe. Such techniques may prompt a scent delivery system to beginproducing the scent before the keyframe is shown or heard. For example,the lead time could be measured by duration (e.g., 50 milliseconds) orframe count (e.g., 5 frames).

Thus, unscented digital content could be manually repurposed intoscented digital content by an administrator and/or automaticallyrepurposed into scented digital content by the repurposing engine. Therepurposing engine may also engage in machine learning to identifycandidate instances and keyframes, identify the appropriate scent(s) tobe produced at the keyframes, etc. For example, the repurposing enginemay monitor the overall effectiveness of a scent marketing campaign byanalyzing different quantitative metrics, such as the merchant's retailtraffic, click-through rates, conversion rates, etc. Such techniquesallow the repurposing engine to identify the characteristics (e.g.,scent type, scene type, and content type) that correspond with effectivescent marketing campaigns, and then use this information to moreintelligently produce scents that align with content. For example, therepurposing engine may discover that scented advertisements for aperfume product are much more effective when shown after romance filmsrather than action films or comedy films.

Various machine learning algorithms and techniques could be employed bythe repurposing engine, including Naïve Bayes Classifier algorithms, KMeans Clustering algorithms, Support Vector Machine algorithms, linearregression, logic regression, artificial neural networks, etc. Thesemachine learning algorithms/techniques may be chosen based onapplication (e.g., supervised or unsupervised learning) and optimizedbased on whether the administrator has confirmed candidate instancesproposed by the repurposing engine, indicated that a certain scentcreated a more realistic or desirable experience, indicated that acertain scent led to more effective advertising campaigns (e.g.,measured by increased viewer engagement or increased product sales),etc.

Note that the tasks performed by the repurposing engine could also beperformed by a series of dedicated modules. For example, a processingmodule may be responsible for identifying candidate instances, a stimulicreation module may be responsible for identifying the scentedmaterial(s) needed to recreate a certain scent, and an analytics modulemay be responsible for monitoring the effectiveness of a scent marketingcampaign.

Terminology

Brief definitions of terms, abbreviations, and phrases used throughoutthe specification are given below.

Reference in this specification to “one embodiment” or “an embodiment”means that a particular feature, structure, or characteristic describedin connection with the embodiment is included in at least one embodimentof the disclosure. The appearances of the phrase “in some embodiments”are not necessarily referring to the same embodiments, nor are theynecessarily referring to separate or alternative embodiments that aremutually exclusive of one another. Moreover, various features aredescribed that may be exhibited by some embodiments but not others.Similarly, various requirements are described that may be requirementsfor some embodiments but not others.

Unless the context clearly requires otherwise, the words “comprise” and“comprising” are to be construed in an inclusive sense rather than anexclusive or exhaustive sense (i.e., in the sense of “including, but notlimited to”). The terms “connected,” “coupled,” or any variant thereofincludes any connection or coupling, either direct or indirect, betweentwo or more elements. The coupling or connection between the elementscan be physical, logical, or a combination thereof. For example, twodevices may be coupled directly to one another or via one or moreintermediary channels/devices. Devices may also be coupled in such a waythat information can be passed there between, despite not sharing anyphysical connection with one another. The words “associate with,”meanwhile, mean connecting or relating objects, items, etc.

Where the context permits, words used in the singular sense or theplural sense may also be used in the plural sense or the singular sense,respectively. The word “or,” in reference to a list of two or moreitems, covers all of the following interpretations of the word: any ofthe items in the list, all of the items in the list, and any combinationof the items in the list. If the specification states a component orfeature “may,” “can,” “could,” or “might” be included or have acharacteristic, that particular component or feature is not required tobe included or have the characteristic in every embodiment.

The term “module” refers broadly to software, hardware, and/or firmwarecomponents. Modules are typically functional components that cangenerate useful data or other output using specified input(s). A modulemay or may not be self-contained. A software program or application mayinclude one or more modules.

The terminology used in the Detailed Description is intended to beinterpreted in its broadest reasonable manner, even though it is beingused in conjunction with certain embodiments. The terms used in thisspecification generally have their ordinary meanings in the art, in thecontext of the disclosure as a whole and in the specific context whereeach term is used. For convenience, certain terms may be highlightedusing, for example, capitalization, italics, and/or quotation marks.However, the use of highlighting has no influence on the scope andmeaning of a term. The scope and meaning of a term is the same, in thesame context, whether or not it is highlighted.

Consequently, although alternative language and synonyms may be used forsome terms, special significance is not to be placed upon whether or nota term is elaborated or discussed herein. A recital of one or moresynonyms does not exclude the use of other synonyms. The use of examplesanywhere in this specification, including examples of any termsdiscussed herein, is intended to be illustrative only. These examplesare not intended to limit the scope and meaning of the disclosure or ofany exemplified term. Likewise, the disclosure is not limited to variousembodiments described below.

System Overview

FIGS. 1A-B depict conventional head-mounted displays (HMDs) 100 a-b thateach include an image display system 102 a and an optical displaysurface 104 a-b configured to reflect or emit light. Some conventionalHMDs are designed so that visual content emitted by the image displaysystem 102 a is reflected by the optical display surface 104 a-b towardsthe user's eyes. Other conventional HMDs are designed so that visualcontent created/processed by the image display system 102 a is projecteddirectly into the user's eyes by the optical display surface 104 a-b.

Conventional HMDs 100 a-b can cover one or both of the user's eyes. Morespecifically, binocular HMDs present a separate image to each of theuser's eyes, while bi-ocular HMDs present a single image to both of theuser's eyes, and monocular HMDs present a single image to only one ofthe user's eyes. However, as shown in FIGS. 1A-B, conventional HMDs 100a-b do not cover the user's nose or mouth, which separates or isolatesseveral critical senses. For example, the HMDs 100 a-b shown hereisolate the olfactory sense and the gustatory sense from the visualsense.

Consequently, a user of a conventional HMD will continue to receivestimuli from the real world that can disrupt the realism of theaugmented or virtual reality content shown by the HMD. In fact, some ofthese stimuli may be partially or entirely counter to what the userexpects. For example, the user might smell a fried chicken scent that isemanating from the kitchen while viewing a dissimilar virtualenvironment (e.g., the inside of an spaceship) through an HMD. Unrelatedstimuli are likely to cause the user to have a disjointed andunrealistic experience while using the HMD.

HMDs can be configured to display two different types of digitalimagery. First, some HMDs are configured to display simulated (e.g.,computer-generated) digital images that together form an entirelyvirtual environment. This is referred to as “virtual reality.” Virtualreality generally requires the user wear a helmet, goggles, or someother headwear that forms an enclosed area around the user's eyes,thereby blocking out the real world. Second, some HMDs are configured todisplay simulated (e.g., computer-generated) images that aresuperimposed onto real-world content perceived by the user. This isreferred to as “augmented reality.” Augmented reality allows the user toconcurrently view virtual imagery and real-world imagery and, in someinstances, interact with virtual content that relates to what is seenthe real world. Generally, the user is able to distinguish between thevirtual content and the real-world imagery. For example, augmentationmay take place through superposition of a simulated image on anon-simulated, real-world environment. The simulated image may allow theuser to view additional data relevant to a desired task, thenon-simulated, real-world environment, etc. The simulated image couldalso be adapted to allow for user interaction. However, augmented andvirtual reality experiences are generally limited in how realistic theyare because they are largely visual experiences.

HMD systems fall into three general categories: binocular systems,bi-ocular systems, and monocular systems. Binocular HMD systems presenta separate image to each of the user's eyes, while bi-ocular HMD systemspresent a single image to both of the user's eyes, and monocular HMDsystems present a single image to only one of the user's eyes. Oneskilled in the art will recognize that the systems and techniquesdescribed herein can be adapted for any of these systems. The use of oneor more of these terms in describing an embodiment is for the purpose ofillustration only.

HMDs enable users to more easily interact with augmented or virtualreality assets. The assets can include, for example, network-accessibleinterfaces (e.g., that are accessible through a web browser), softwareprograms/applications, video games, and cinematic videos. HMDs couldalso be designed to facilitate non-visual experiences. For example, manyconventional HMDs include (or can be easily paired with) headphones orspeakers through which audio assets (e.g., audiobooks, concerts, andradio programs) can be projected.

FIG. 2A is an inside view of a binocular HMD 200 that includes a scentdelivery system 202. FIG. 2B is a front view of the binocular HMD 200that includes a scent delivery system 202. The scent delivery systemsdescribed herein enhance the realism of digital content (e.g., visualcontent and non-visual content) experienced by a user by replacingreal-world stimuli with controlled stimuli that are related to thedigital content. For example, the scent delivery systems can increasethe realism of digital content by producing scent(s) that trick theuser's olfactory sense (i.e., sense of smell).

A scent delivery system 202 can be completely self-contained, completelyintegrated into the binocular HMD 200, or partially self-contained andpartially integrated into the binocular HMD 200. In some embodiments,the scent delivery system 202 is detachably connectable to the frame ofthe binocular HMD 200 using one or more fasteners (e.g., magnets,mechanical clips/tracks, or some other quick release mechanism). Thefastener(s) allow the scent delivery system 202 to be readily attachedto, and then removed from, the binocular HMD 200 without the use oftools. For example, the structural frame of the binocular HMD 200 mayinclude features (e.g., cavities) configured to mate with correspondingfeatures (e.g., protrusions) of a scent delivery system 202. In otherembodiments, the scent delivery system 202 is fixedly attached to theframe of the binocular HMD 200 using, for example, screws, nuts andbolts, or an adhesive material/substance.

Additionally or alternatively, some or all of the components that enablescent to be dispensed (e.g., pumps, reservoirs, and/or fans) could beintegrated within the HMD. For example, the pump(s) and reservoir(s) maybe integrated into the structural frame of an HMD (which renders aseparate attachment unnecessary). In such embodiments, the pump(s) maybe arranged such that scented material(s) are ejected toward the user'snose.

One skilled in the art will recognize that embodiments involving abinocular HMD are described herein for the purpose of illustration only.The scent delivery systems and computer-implemented techniques describedherein are equally applicable to monocular HMDs, bi-ocular HMDs, andother computing devices (e.g., mobile phones, televisions, and wearabledevices). The scent delivery system 202 may be customizable for aparticular user, a particular type of experience (e.g., augmented orvirtual reality content), a particular HMD (e.g., Oculus Rift® orMicrosoft Hololens®), etc.

The scent delivery system 202 can be designed so that it partially orentirely covers the user's nose. For example, FIGS. 2A-B depict a scentdelivery system 202 that completely envelops the user's nose. However,the scent delivery system 202 could also be designed so that the base ofthe user's nose is exposed. In some embodiments, some of the components(e.g., the pumps 204, reservoirs 206, and/or fan 208) are integratedinto the structural frame of the binocular HMD 200 so that the binocularHMD 200 appears similar to a conventional HMD (e.g., the HMDs of FIGS.1A-B) despite supporting scent functionality.

More specifically, the scent delivery system 202 can include astructural frame 210, one or more pumps 204, one or more reservoirs 206,and a fan 208. The structural frame 210 can include fasteners that allowthe scent delivery system 202 to be readily connected to, and thenremoved from, the binocular HMD 200. For example, the structural frame210 could include features (e.g., detents or cavities) configured tomate with corresponding features (e.g., protrusions) of the scentdelivery system 202. Examples of fasteners include magnets, mechanicalclips/latches/tracks, and other quick release mechanisms. In otherembodiments, the scent delivery system 202 is fixedly attached to thestructural frame 210 using screws, nuts and bolts, adhesivematerials/substances, etc.

The pump(s) 204 allow a scented material (e.g., liquid or air) stored ineach of the reservoir(s) 206 to be dispensed in a controlled manner. Forexample, the pump(s) 204 could dispense scented liquids in particularquantities, at particular temperatures, with particular forces, etc. Infact, the pump(s) 204 could be collectively controlled by scentcircuitry that can intelligently cause dispersal of a single scentedmaterial and combinations of multiple scented materials. Said anotherway, the pump(s) 204 could sequentially or simultaneously dispense thescented material(s) stored in the reservoir(s) 206.

In some embodiments the pump(s) 204 spray a scented material directlytoward the user's nose, while in other embodiments the pump(s) 204 spraythe scented material within the internal cavity of the structural frame210 (in which case the fan 208 could direct the scented material towardsthe user's nose or away from the user's nose). As further describedbelow, the pump(s) 204 may dispense a small quantity of a single scentedmaterial when a slight aroma is desired, a large quantity of a singlescented material when a strong aroma is desired, small or largequantities of multiple scented materials simultaneously, etc.

Scent could also or instead be delivered via small absorbent pads thatare connected to the reservoir(s) 206 and disposed near the user's nose.For example, one or more absorbent pads could be affixed on the interiorsurface of the structural frame 210 of the scent delivery system 202(i.e., within the internal cavity). Unlike the pump(s) 204 (which areable to controllably dispense scented materials at certain times), theabsorbent pads typically continually dispense small amounts of scentedmaterial.

The reservoir(s) 206 generally retain scented material(s) havingdifferent scents. For example, the three reservoirs depicted in FIGS.2A-B may correspond to general scents (e.g., sweet, savory, and floral),specific scents (e.g., cinnamon, citrus, and rose), or some combinationthereof. Moreover, the reservoirs 206 may be readily replaceable by theuser or some other entity (e.g., the manufacturer of the HMD or aservice representative). In such embodiments, the reservoirs could bereplaced by “capsules” of scented material that are modularlyreplaceable, which enables the user to customize which scent(s) can beproduced by the scent delivery system 202.

Note, however, that one of the reservoir(s) 206 could include anon-scented material (e.g., water) or a neutrally-scented material thatis controllably dispensed within the internal cavity to neutralize otherscents. For example, a neutralizing scent (also referred to as a“cleansing scent”) may be created to eliminate strong scents (e.g., asmoke odor or a food-based odor) or unwanted scents (e.g., a skunk odor)the user is unlikely to want to linger. The neutrally-scented materialcould also include particles (e.g., natural oils) that eliminate scentsby attaching to odor molecules to form a neutral compound.Neutrally-scented material(s) are often circulated by the fan 208 tomore quickly flush scent(s) from the internal cavity of the structuralframe 210. Additionally or alternatively, the scent delivery system 202could produce a masking scent that overwhelms strong scents or unwantedscents. The masking scent could be, for example, peppermint, spearmint,or anise.

In some embodiments, some or all of the material(s) housed within thereservoir(s) 206 may be functional in nature. One example of afunctional material is a scented liquid that includes pheromone(s) thattrigger specific response(s), such as alarm pheromones, signalpheromones, sex pheromones, etc. For example, a scent delivery system202 could dispense a scented or unscented material that includes analarm pheromone during a horror film or a horror video game.

In some embodiments, the scented material(s) stored in the reservoir(s)206 also include a flavor compound having a particular taste.Accordingly, the user may experience digital content using her visual,olfactory, and/or gustatory senses. For example, when the user views adigital environment that includes a citrus grove, the scent deliverysystem 202 may dispense a material having a citrus scent and/or a citrustaste. Scent and taste could be facilitated by a single material ormultiple materials. For example, the scent delivery system 202 couldinclude one reservoir that stores a material having a scent-enablingcompound and another reservoir that stores a material having ataste-enabling compound. As another example, the scent-enabling compoundand the taste-enabling compound may both be included in the samematerial that is stored in a single reservoir.

Several examples of scent-enabling compounds (also referred to as “aromacompounds”) are provided in Table I, which is not intended to beexhaustive.

TABLE I Examples of aroma compounds. Compound Name Fragrance NaturalOccurrence Geranyl acetate Fruity, rose Rose, Floral Methyl formateEthereal — Methyl acetate Sweet, solvent, nail polish — Methylpropionate Sweet, fruity — Methyl propanoate Methyl butyrate Fruity,apple, pineapple Pineapple Methyl butanoate Ethyl acetate Sweet, solventWine Ethyl butyrate Fruity, orange, pineapple — Ethyl butanoate Isoamylacetate Fruity, banana, pear Banana plant Pentyl butyrate Fruity, pear,apricot — Pentyl butanoate Pentyl pentanoate Fruity, apple — Octylacetate Fruity, orange — Benzyl acetate Fruity, strawberry StrawberryMethyl anthranilate Fruity, grape — Myrcene Woody Verbena, bay leafGeraniol Rose, flowery Geranium, lemon Nerol Sweet rose, flowery Neroli,lemongrass Citral Lemon Lemongrass Citronellol Lemon Lemongrass, rose,pelargonium Linalool Floral, sweet, woody, Coriander, sweet basil,lavender lavender Nerolidol Woody, bark Neroli, ginger, jasmine LimoneneOrange Orange, lemon Camphor Camphor Camphor laurel Menthol Menthol,peppermint Mentha (corn mint), peppermint Carvone Caraway, spearmintCaraway, dill, spearmint Terpineol Lilac Lilac, cajuput Alpha-IononeViolet, woody Violet Thujone Minty Wormwood, lilac, juniper EucalyptolEucalyptus Eucalyptus Benzaldehyde Almond Bitter almond Eugenol CloveClove Cinnamaldehyde Cinnamon Cinnamon, cassia Ethyl maltol Cookedfruit, caramelized — sugar Vanillin Vanilla Vanilla Anisole Anise AniseAnethole Anise Anise, sweet basil Estragole Tarragon Tarragon ThymolThyme Thyme Trimethylamine Fishy, ammonia — Putrescine Rotting fleshRotting flesh Diaminobutane Cadaverine Rotting flesh Rotting fleshPyridine Fishy Belladonna Indole Fecal, flowery Feces, jasmine SkatoleFecal Feces, orange blossoms Furaneol Strawberry, pineapple, Strawberry,cotton candy, sweet pineapple, buckwheat, tomato Hexanol Grass, woody —Anisic aldehyde Sweet, chocolate, vanilla, — strawberry, raspberry,apricot

The fan 208 could be set to continuously or periodically circulate airaround the user's nose. For example, the fan 208 may circulate air whenthe scent delivery system 202 begins dispensing scented material(s) orwhen the user enables the fan 202 (e.g., by turning a fan setting to“on”). In some embodiments, the structural frame 210 includes one ormore openings (also referred to as “breaks” or “outlets”) that allow airto more easily circulate through the internal cavity of the structuralframe 210. These opening(s) may be strategically positioned with respectto the user's noise (e.g., near the nostrils as shown in FIGS. 2A-B) orwith respect to the pump(s) 204 and/or fan 208.

The fan 208 may also be used to modify the airflow in or around theinternal cavity of the structural frame 210. Different attributes (e.g.,acceleration, velocity, and temperature) could be modified to simulatewind that is included in content presented by the binocular HMD 200. Forexample, the fan 208 may intelligently switch between several fansettings to simulate different airflows while the user views anadvertisement for a vehicle. The fan 208 may perform such actions incombination with the pump(s) 204. For example, the fan 208 may generatea high airflow and a pump may dispense a material having a salty flavor(e.g., a salty smell and/or a salty taste) while the binocular HMD 200presents appropriate content, such as a cinematic video involving theocean or an advertisement for a golf course near the coast.

In some embodiments, the binocular HMD 200 also includes otherstructural features or components that enhance the user experience. Forexample, the binocular HMD 200 may include one or more sensors 212 thatare positioned proximate to the user's face. The sensor(s) 212 canmeasure skin temperature, temperature within the internal cavity,humidity, motion, etc. Scent circuitry 214 may be responsible formodifying which scented material(s) are dispensed and how much of eachscented material is dispensed based on signals received from each of thesensor(s) 212. For example, if sensor data indicates that the skintemperature has begun to rise or the user has begun to sweat, the scentcircuitry 214 may lessen the amount of scented material dispensed byeach pump to avoid overstimulation of the user's senses. As anotherexample, if the sensor data indicates that the user has begun to moveher head toward the scent source (e.g., toward a flower in a virtualenvironment), the scent circuitry 214 may increase the amount of scentedmaterial dispensed by each pump to increase scent intensity.

The scent circuitry 214 (which may also be referred to as an“electronics module”) can process data generated by each of thesensor(s) 212 and optimize the delivery of scented material(s) by thepump(s) 204. The scent circuitry 214 can include a power supply (e.g., abattery or an electrical coupling to the power supply of the HMD),memory, one or more processors, and/or a communication module thatallows the scent delivery system 202 to communicate with the binocularHMD 200 and/or another computing device (e.g., a mobile phone, tablet,personal computer, or server). The communication module may allow thescent delivery system 202 to communicate over the Internet, a local areanetwork (LAN), a wide area network (WAN), a point-to-point dial-upconnection, a cellular network, etc. Additionally or alternatively, thecommunication module may communicate via a short-range wirelessprotocol, such as Wi-Fi, near-field communication (NFC), radio-frequencyidentification (RFID), or Bluetooth.

Scented material(s) are preferably dispensed by the pump(s) 204responsive to the content is being presented by the binocular HMD 200.For example, the scent circuitry 214 may specify that a floral scentshould be produced when a flower is shown in a cinematic video. Asanother example, the scent circuitry 214 may specify that a pine scentshould be produced when a character in non-visual content (e.g., anaudiobook) enters a forest. The scent circuitry 214 (as well as theother components of the scent delivery system 202) may be poweredthrough a wired or wireless medium. For example, in some embodiments thescent circuitry 214 includes an internal power supply (e.g., arechargeable battery or a single cell (i.e., button cell) battery),while in other embodiments the scent circuitry is electrically coupledto, and powered by, the binocular HMD 200.

Some embodiments of the binocular HMD 200 also include one or moreheating elements 218 that are disposed near the reservoir(s) 206,pump(s) 204, and/or fan 208. The heating element(s) 218 can be used todispense the materials stored in the reservoir(s) 206 at certaintemperatures or in certain forms. For example, a heating element couldbe used to produce steam from water stored in a reservoir to increasethe humidity within the internal cavity of the structural frame 210. Theheating element(s) 218 could also be used to heat an airflow created bythe fan 208. For example, a warm airflow may be generated when a desertenvironment is shown in a cinematic video.

One or more sensors 216 could also be disposed on the outer surface ofthe scent delivery system 202 and/or the binocular HMD 200. Thesensor(s) 216 may monitor various characteristics of the ambientenvironment. For example, the sensor(s) 216 may include a camera thatcaptures the user interactions with the ambient environment, a lightsensor that monitors ambient illuminance levels, an audio sensor thatsenses vocal commands, etc. One skilled in the art will recognize thatother types of sensors could also be included. The type of sensor(s) 216could be chosen based on whether the HMD system is adapted for aparticular application. For example, HMDs designed for outdoor use mayinclude a temperature sensor that generates temperature data and a windsensor that generates wind data. The temperature data and/or the winddata may be used to determine whether additional scented material needsto be dispensed to counteract a windy environment.

Unlike traditional scent dispersal systems (e.g., the Smell-O-Vision orthe Smellitizer), the technologies introduced here are readily portableand allow scents to be delivered directly to an individual experiencingdigital content. Said another way, traditional scent dispersal systemsare designed to produce scents in a single physical location, while thescent delivery systems described herein can be used regardless of wherethe individual decides to experience the digital content. Moreover,because the scent can be delivered to a given individual in a targetedmanner, other individuals who are nearby will not able to smell thescent (and are thus undisturbed). Consequently, multiple individuals canexperience realistic digital content in close proximity to one anotherdespite experiencing different digital content or having differentdelivery preferences. For example, some individuals may enable the scentfunctionality while other individuals may opt not to participate inscented experiences.

As shown in FIG. 2A, in some embodiments the binocular HMD 200 includesone or more lighting elements 220 that are installed near the edges ofthe optical display surface. The lighting element(s) 220 are generallyadjustable, thereby enabling the binocular HMD 200 to establish acontrolled lighting environment. By intelligently illuminating thelighting element(s) 220, the binocular HMD 200 may make the user believeas though the optical display surface is larger than it truly is. Forexample, the lighting element(s) 220 could serve as a bias light orbacklight by illuminating the periphery of the user's vision based onthe visual content being shown (e.g., using a color that matches orcomplements the visual content). The lighting element(s) 220 couldproduce white light and/or colored light. The lighting element(s) 220could also be used to further increase the realism of visual contentpresented by the binocular HMD 200 in several different respects. Forexample, the lighting element(s) 220 could provide additional sensorystimuli (e.g., bright bursts of light in the user's peripheral visionduring an action movie or shooter video game).

FIG. 3 depicts an example of a network environment that includes an HMD302 having a scent delivery system, a content server 304, and anothercomputing device 306 (here, a television and game console). The HMD 302,content server 304, and/or computing device 306 can be connected via oneor more computer networks 308 a-c, which may include the Internet, localarea networks (LANs), wide-area networks (WANs), metropolitan areanetworks (MANs), cellular networks, etc. Additionally or alternatively,the HMD 302, content server 304, and/or computing device 306 maycommunicate with one another over a short-range communication protocol,such as Bluetooth, near-field communication (NFC), etc. Examples ofcomputing devices 306 include mobile phones, tablet computers, laptopcomputers, personal computers, network-connected (“smart”) homeappliances (e.g., televisions and speakers, such as Amazon Echo® andGoogle Home®), game consoles (e.g., Sony PlayStation® or Microsoft Xbox®game console), mobile gaming devices (e.g., Sony PSP®),television-connected devices (e.g., Roku®, Apple TV®, GoogleChromecast®, Amazon FireStick®, Android TV®, Blu-ray Disc™ players),other network-connected devices having an interface (e.g., fitnesstrackers, Apple iPod Touch®, Apple Watch®), etc.

Generally, the HMD 302 is responsible for generating and presentingcontent to a user. While the content is typically in the form ofaugmented or virtual reality content, other forms of content (e.g.,visual content such as cinematic videos and real-time video feeds, ornon-visual content such as audiobooks and music) could also be presentedto the user. The content server 304 and/or the computing device 306 canbe coupled, wired or wirelessly, to the HMD 302. For example, the HMD302 may be communicatively coupled to the computing device 306 via anwired digital-audio connection (e.g., High-Definition MultimediaInterface (HDMI) or fiber optic), antenna, wireless communicationmodule, etc. In some embodiments the content server 304 and/or thecomputing device 306 are responsible for delivering content to the HMD302, while in other embodiments the HMD 302 stores some or all of thecontent locally (e.g., within a memory).

When the HMD 302 is configured to generate virtual reality content, itmay only be communicatively coupled to the content server 304. However,when the HMD 302 is configured to generate augmented reality content, itcould be communicatively coupled to the content server 304 and/or thecomputing device 306. Consequently, the user of the HMD 302 may be ableto see augmented reality content when viewing video, listening to audio,or playing video games using the computing device 306. The computingdevice 306 may also be responsible for presenting content to the userwhile the HMD 302 simply enhances the realism of the content bygenerating sensory stimuli (e.g., scents, tastes, and/or visualeffects). Thus, the HMD 306 may improve the user experience withoutpresenting content itself.

As noted above, in order to make these experiences more realistic, theHMD 302 (and, more specifically, a scent delivery system) may beconfigured to produce one or more scents at particular times. Forexample, the HMD 302 could dispense a floral scent when a digital gardenenvironment is viewable through the HMD 302. As another example, the HMD302 could dispense a vanilla scent and a cinnamon scent when augmentedreality content is viewable through the HMD 302 that augments a cookingshow shown on a television in the real world. Further yet, the HMD 302could dispense a pine scent, a grass scent, or a woody scent when a golftournament is shown on a television in the real world. In fact, the HMD302 may switch between these scents as golfers play different holes indifferent environments. The HMD 302 may produce scents regardless ofwhether it is responsible for presenting content to the user.

The strength, direction, and frequency of the sensory stimuli producedby the HMD 302 generally correspond to features of the content presentedto the user by the HMD 302 and/or the other computing device 306. Insome embodiments the instructions for producing the sensory stimuli areproduced by a source of the content, while in other embodiments theinstructions for producing the sensory stimuli are created automatically(e.g., by the HMD 302, content server 304, or other computing device306) after analyzing the content to identify elements that areindicative of a sensory stimuli source. The instructions may be embeddedwithin the content (e.g., encoded in a video track, audio track, and/ormetadata) or included as a distinct instruction set that accompanies thecontent and is separately executable by the HMD 302.

In some embodiments, the scent delivery system is housed within anetwork-connected mask (or some other form of headwear) that allows theindividual to view visual content presented by another network-connectedcomputing device, such as a television. The network-connected mask andthe network-connected computing device may communicate with one anothervia a short-range wireless communication technology/protocol, such asWi-Fi, Bluetooth, NFC, cellular radio, infrared, RFID, etc.

FIG. 4 depicts a scent delivery system 400 that can be detachablyconnected to an HMD configured to display augmented or virtual realitycontent. In some embodiments, the scent delivery system 400 includes acompartment 404 that is able to hold one or more reservoirs 402 ofscented material (e.g., air or liquid). Each scented material maycorrespond to a separate “capsule” that is modularly replaceable. Saidanother way, the capsules can be easily installed within, and removedfrom, the scent delivery system 400. Each capsules may correspond to ageneral scent or a specific scent.

As shown in FIG. 4, the reservoirs 402 are often retained within acompartment 404 that is readily accessible to the user while the scentdelivery system 400 is attached to an HMD. For example, the scentdelivery system 400 may include a door 406 that can be securely closedvia fasteners 408, such as magnets, mechanical clips/tracks, an adhesivematerial/substance, etc. The door 406 enables the user to replace thereservoirs 402 in response to determining new scents are desired ornecessary for content that is to be presented by the HMD. For example,the user may wish to replace some or all of the reservoirs 402 prior towatching a cinematic video, playing a video game, or interacting withaugmented or virtual reality content. Note, however, that in someembodiments the reservoirs 402 are only accessible when the scentdelivery system 400 is separated from the HMD.

The scent delivery system 400 may also dispense scented materials fromthe reservoirs 402 in order to affect the user's sense of taste. HMDsthat show virtual reality content prevent users from observing what theyare eating or drinking while viewing the virtual reality content.However, there is a strong physiological connection between differentsources of sensory information. For instance, the ability to recognizeflavors is often affected by visual information and olfactoryinformation. The technology described herein can make use of thesephysiological connections to trick the user's senses.

For example, a user could be tricked into believing that water is asweeter drink by having the HMD present a digital environment thatincludes a sweet drink and/or by having the scent delivery system 400produce a sweet scent. Similarly, the user could be tricked intobelieving a wine is much stronger or better than it actually is bypresenting a picturesque vineyard and/or dispensing an appropriatesmell. Scent delivery systems can be used to make bland foods seem moresalty, spicy, bitter, sweet, or sour by modifying the content presentedto the user and/or the scent(s) produced by the scent delivery system400.

As noted above, the scent delivery system 400 could also store materialsthat include a flavor compound having a particular taste. Consequently,visual, olfactory, and/or gustatory experiences can be selectivelymodified in order to improve the user experience. For example, goodflavors may be enhanced, while bad flavors may be overwhelmed orreplaced entirely. In some embodiments the scent delivery system 400includes separate reservoirs for storing materials having ascent-enabling compound and materials having a taste-enabling compound,while in other embodiments the scent-enabling compound and thetaste-enabling compound are included in the same material that is storedin a single reservoir.

Implementation Overview

Scented paper products (e.g., fragrance strips) have long been used toincrease consumer interest in products such as cosmetics, perfumes,colognes, soaps, shampoos, personal hygiene items, etc. Scent deliverysystems are introduced here that are able to bring scentedadvertisements into other forms of content, including visual content andnon-visual content. Such technology can be used to address a fundamentalchallenge that is inherent in digital transactions (i.e., an inabilityto accurately gauge different characteristics of a product) by engaginga consumer's senses. More specifically, scent delivery systems can beused to increase the effectiveness of marketing, advertising, andbranding campaigns by controllably producing certain scent(s).

FIG. 5 depicts a development platform 500 that allows content (e.g.,visual content and non-visual content) to be specially designed forpresentation by a computing device that is connected to a scent deliverysystem. In such embodiments, a processor can determine whether todispense a scented material based on instructions created by anadministrator 512. The administrator 512 may be associated with amerchant (i.e., manufacturers/sellers of goods/services), a fragrancecompany (i.e., manufacturers of scented materials), or a developmentcompany (e.g., manufacturers of scent delivery systems and producers ofdigital content). As noted above, the administrator 512 may generate theinstructions by working with one or more of these entities to identifythe one or more scented materials needed to accurately reproduce acertain scent. The instructions may also specify a delivery timestamp,an intensity/quantity of each scented material, etc. In some embodimentsthe instructions are embedded within the digital content (e.g., encodedin a video track, audio track, and/or metadata), while in otherembodiments the instructions are included in a distinct instruction setthat accompanies the digital content and is separately executable by theprocessor. Similar techniques could also be used for visual content(e.g., applications, games, and cinematic videos) and non-visual content(e.g., audiobooks, concerts, and radio programs).

Older, unscented digital content could also be repurposed for use with acomputing device that is connected to a scent delivery system. Anadministrator 512 may be able to tag unscented digital content to createkeyframes that cause the scent delivery system to produce scent(s). Forexample, the administrator 512 may identify keyframes in a directadvertisement that will benefit from the production of a scent. Asanother example, the administrator 512 may identify keyframes inexisting digital content (e.g., a cinematic film) in order to create anindirect advertisement.

Content modifications may be completed using a graphical user interface(GUI) 514 that is supported by a repurposing engine 500. Morespecifically, the administrator 512 could upload unscented digitalcontent to the GUI 514, which prompts the repurposing engine 500 toanalyze the unscented digital content to determine whether scent(s)should be produced and, if so, which scent(s) should be produced. Insome embodiments, the repurposing engine 500 automatically identifiescandidate instances for dispensing scented material(s). The candidateinstances typically include one or more features (e.g., a character or aproduct, such as a perfume bottle or a piece of fruit) that are readilyrecognizable upon performing content analysis. Content analysis mayrequire the repurposing engine 500 execute image processing algorithms,audio processing algorithms, etc. For example, the repurposing engine500 may perform image segmentation (e.g., thresholding methods such asOtsu's method, or color-based segmentation such as K-means clustering)on individual frames of the digital content to isolate regions andobjects of interest. As another example, the repurposing engine 500 mayperform audio segmentation on an audio track of the digital content toidentify segments that likely correspond with certain events. Forinstance, significant variations in tempo, pitch, or intensity may beindicative of an event that would benefit from additional sensorystimuli.

In some embodiments, the repurposing engine 500 automatically determinescharacteristics regarding the production of each scent. Thecharacteristics could include how much scented material should bedispensed (e.g., whether the aroma should be faint, medium, or strong),when the scent material should be dispensed, whether a combination ofmultiple scented materials is necessary to produce a certain scent, etc.The administrator 512 may be able to modify the instances and/orcharacteristics proposed by the repurposing engine 500. Theadministrator 512 may also be able to specify additional instancesand/or characteristics using the GUI 514.

Thus, unscented digital content could be manually repurposed intoscented digital content by an administrator 512 and/or automaticallyrepurposed into scented digital content by the repurposing engine 500.The unscented digital content could be non-visual content or visualcontent, such as two-dimensional (2D) content or three-dimensional (3D)content. The repurposing engine 500 may also engage in machine learningto identify candidate instances and keyframes, identify the appropriatescent(s) to be produced at the keyframes, etc. For example, therepurposing engine 500 may monitor the overall effectiveness of a scentmarketing campaign by analyzing different quantitative metrics, such asthe merchant's retail traffic, click-through rates, conversion rates,etc. Such techniques allow the repurposing engine 500 to identify thecharacteristics (e.g., scent type, scene type, and content type) thatcorrespond with effective scent marketing campaigns, and then use thisinformation to more intelligently produce scents that align withcontent. For example, the repurposing engine 500 may discover thatscented advertisements for a perfume product are much more effectivewhen shown after romance films rather than action films or comedy films.

The repurposing engine 500 includes one or more processors 502 forexecuting these tasks and one or more storage modules 504 for storingthe relevant data (e.g., digital content, ground truth feature sets foridentifying candidate instances, known combinations of scentedmaterials). The tasks performed by the repurposing engine 500 could alsobe performed by a series of dedicated modules. For example, a processingmodule 506 may be responsible for identifying candidate instances, astimuli creation module 508 may be responsible for identifying thescented material(s) needed to produce a certain scent, and an analyticsmodule 510 may be responsible for monitoring the effectiveness of ascent marketing campaign, noting user reactions to different scents,etc.

Disjointed experiences will often occur if scented are produced tooearly or too late. Accordingly, the repurposing engine 500 may employpredictive analytics to determine when, exactly, a scent should beproduced for a given keyframe. FIG. 6A depicts an example of a videotrack that includes a keyframe identified through image segmentation.FIG. 6B, meanwhile, depicts an example of an audio track that includes akeyframe identified through audio segmentation. As shown here,predictive analytics models may prompt a scent delivery system to beginproducing a scent before the keyframe is shown or heard. For example,the lead time could be measured by duration (e.g., 50 milliseconds) orframe count (e.g., 5 frames). In some embodiments, the lead time isbased on a user characteristic known to affect sensory sensitivity, suchas age, gender, relevant ailments (e.g., infections and injury), smokingstatus, etc.

However, the lead time could also vary based on other factors, such asthe type of digital content, the duration of usage of the scent deliverysystem, etc. For example, the lead time may increase proportional to theduration of usage. Long usage durations generally correspond todecreases in sensory sensitive and increases in reaction time (i.e., theelapsed time between the presentation of a sensory stimulus and thesubsequent behavioral response). As another example, the lead time mayincrease for some types of digital content (e.g., visual content wheremultiple senses are engaged) and decrease for other types of digitalcontent (e.g., non-visual content where only the sense of hearing isengaged). Predictive analytics models may also be used to determine whenthe scent delivery system should cease production of scent, airflow,heat, etc.

Keyframes may be used to indicate when various events should occur. FIG.6A, for example, illustrates how a keyframe may prompt a scent deliverysystem to perform a single action (e.g., eject a scented material).However, a keyframe could also induce multiple actions. FIG. 6B, forexample, illustrates how a keyframe may prompt a scent delivery systemto begin ejecting a scented material, begin circulating air, ceaseejecting the scented material and cease circulating air. An individualkeyframe could also be used to induce each event such that multipleactions correspond with multiple keyframes, though keyframes could beshared.

FIG. 7 depicts an interface through which a user can browse itemsoffered for sale by an electronic commerce platform, which could rangein size from large retailers (e.g., Amazon.com®) that sell a wideassortment of products to small retailers that sell a single product.The interface may be viewed by the user on an HMD or some othercomputing device (e.g., computing device 306 of FIG. 3). A scentdelivery system that is communicatively coupled to the HMD or the othercomputing device may enable scent-based browsing of the items offeredfor sale.

For example, a user may execute a search for perfumes or colognesthrough the interface, and then browse different items returned in thesearch result. Similarly, the user might visit a virtual perfume shop orvirtual cologne shop where the user is able to view and smell differentitems offered for sale. As another example, the user could browse scentsand purchase a scent (e.g., as part of a gift that is paired with acinematic movie, digital card, etc.) that will be experienced by theuser or another user.

A scent delivery system may permit the user to test the scents ofdifferent physical items (e.g., perfumes or colognes) before placing anonline order for a physical item. Said another way, a scent deliverysystem may enable scent-based browsing of products that are availablefor purchase by the user. Although the scent of a perfume or cologne maynot be replicated exactly, the scent(s) produced by a scent deliverysystem enable the user to have a better understanding of what she isbuying (e.g., whether a perfume or cologne is intended to have a sweetfragrance, a floral fragrance, a woody fragrance, etc.).

As noted above, scent delivery systems produces scent(s) upon executinginstructions that correspond to the content being experienced by theuser. The instructions could be initially created in several differentways. For example, a manufacturer (e.g., a perfume manufacturer) mayselect one or more scents from a list of scents that are commonlyproduced by scent delivery systems. As another example, instructionscould be automatically created by parsing a product listing to identifytag words that are used to describe the item. For example, a repurposingengine (e.g., repurposing engine 500 of FIG. 5) may execute speechrecognition techniques, such as keyword spotting, to detect tag word(s)that may be used to identify the scent(s) that are most appropriate. Oneskilled in the art will recognize that these different techniques forcreating instructions enable different levels of control in producingscents and accuracy in replicating aromas. For instance, instructionsprovided by the manufacturer of the item are generally more accuratethan instructions that are automatically created based on availableinformation about the item.

FIG. 8A depicts a digital kitchen environment that is viewable throughan HMD. FIG. 8B, meanwhile, depicts a digital garden environment that isviewable through an HMD. One skilled in the art will recognize thatthese digital environments have been selected for the purpose ofillustration only.

A scent delivery system that is connected to the HMD can be configuredto dispense certain scented material(s) as the user of the HMD traversesthe digital environments. These digital environments may be traversedover the course of a cinematic film, video game, etc. For example, asthe user navigates the digital kitchen shown in FIG. 8A and approachesthe digital oranges, the scent delivery system may produce a citrusscent. As another example, as the user navigates the digital gardenshown in FIG. 8B and approaches the digital lilac bush, the scentdelivery system may produce a lilac scent or a floral scent. Thesescents may be produced before the user is able to visualize thecorresponding digital features.

Much like fragrance strips that are placed inside magazines, thesescent(s) can be used as part of a scent-based advertising campaign.Several different use cases illustrate the benefits of enhancing therealism of digital content (e.g., visual content and non-visual content)by replacing unrelated real-world stimuli with controlled stimuli.

-   -   Direct Advertising: A scent delivery system can produce an        appropriate scent while an individual watches an advertisement        for an item (e.g., a perfume or a deodorant). The individual may        also be permitted to buy the item in real time during the        advertisement. For example, if the individual exhibits an        interest in the scent (e.g., by moving her head closer to the        digital source of the scent), the individual may be prompted to        decide whether she would like to order the item directly through        the advertisement. That is, the individual may be able to        complete a digital transaction without disrupting the current        experience by navigating to a separate channel.    -   Indirect Advertising: Scent(s) related to digital content        experienced by an individual can be intelligently replicated by        a scent delivery system, which allows products already present        in the digital content to be indirectly advertised to consumers.        For example, the scent delivery system may produce a perfume        scent when a female character in a cinematic film approaches the        camera. The individual may be able to order the item in real        time (i.e., as the digital content continues to run) and/or save        a product summary for subsequent ordering. Moreover, a list of        product(s) indirectly advertised throughout the runtime of the        digital content could be shown to the individual following the        conclusion of the digital content. In such instances, the        individual may prompt the scent delivery system to reproduce        scent(s) that were previously produced during the runtime of the        digital content. Indirect advertising techniques are        particularly useful for those forms of content where direct        advertising is undesirable (e.g., due to content flow        disruptions) or difficult to carry out (e.g., due to the        presence of ad blockers).    -   Business Model Improvements: Merchants can work with fragrance        companies and development companies to re-create scents that        will improve the effectiveness of advertising campaigns, the        desirability of products, etc. For example, some or all of these        entities may work together to develop a scented advertisement        and generate instructions for producing the appropriate scent(s)        at the appropriate time(s).    -   Promotional Cartridges: Merchants could send promotional        capsules that include scented materials out on a periodic (e.g.,        weekly or monthly) basis. For example, a perfume manufacturer        may distribute promotional capsules each time a new perfume is        developed. As another example, the perfume manufacturer may        distribute promotional capsules as part of the concept testing        process that typically occurs prior to production (e.g.,        multiple capsules could be delivered to potential consumers who        are asked to identify their favorite scent or rate the various        scents).

HMDs often employ an optical positional tracking technique (e.g.,inside-out tracking and outside-in tracking) to determine the positionof certain objects, such as a user's hands and feet. Accordingly,certain scent(s) may be triggered by the user as she navigates thedigital environment. For example, a scent delivery system may produce acitrus scent responsive to a determination that the user has reached outher hand and grasped at the digital oranges. As another example, a scentdelivery system may produce a strawberry scent responsive to adetermination that the user has stepped on a digital strawberry.

FIG. 9 includes an overhead map illustrating how a digital environmentmay include one or more scented regions. The overhead map shown herelargely corresponds to the digital kitchen shown in FIG. 8A. As shownhere, the scented region(s) will often vary in size and/or shape. Insome embodiments these parameters are automatically determined based onthe scent type or certain user characteristics (e.g., knownsensitivities to certain scent types), while in other embodiments theseparameters are manually specified by an administrator who is responsiblefor identifying the scented features. Thus, users may be able to smellitems before those items are shown or described (which can be used asanother form of indirect advertising).

Some of the scented region(s) may be locked in a static (i.e., fixed)location, while other scented region(s) may move throughout the digitalenvironment. For example, the perfume-scented region may move as thedigital person traverses the digital kitchen environment. Consequently,scented regions may occasionally overlap. A scent delivery system willtypically produce both scents in such instances, although the scentdelivery system may also produce a single scent in some instances (e.g.,where one scent is likely to overpower another).

Scented region(s) may also be accessible for a specified duration oftime. For example, the orange-scented region may disappear if thedigital oranges are eaten or thrown in the trash. As another example,the cinnamon-scented region may disappear upon determining the digitalcinnamon rolls have been shown for a certain period of time. Therefore,an individual may smell different scents while traversing a digitalenvironment. The individual may even begin to smell items before thoseitems are viewable within the digital environment. Consequently,different scents may be used to guide the individual through the digitalenvironment. For example, a favorable scent may attract to theindividual to a certain room within the digital environment, and theintensity of the favorable scent may increase as the individual movescloser to the certain room.

Although the overhead map of FIG. 9 represents a digital kitchenenvironment, the same techniques are equally applicable to other digitalenvironments. For example, the technology introduced here may allow anindividual to traverse a digital recreation of a merchant's physicalstore and smell different products that are offered for sale by themerchant. Smell has been shown to be particularly effective in marketingproducts such as cosmetics, perfumes, colognes, soaps, shampoos,personal hygiene items, etc.

If an individual indicates an interest in a product (e.g., by moving herhead closer to the digital source of the scent), the individual may beasked whether she wishes to purchase the product. This can occur inseveral different ways. For example, as shown in FIG. 10A, an orderinterface may be presented by the computing device (e.g., an HMD or anetwork-connected television) that is used to view the visual content.The order interface may prompt the user to approve a purchase of theproduct, input financial information (e.g., name, address, and paymentcard details), decline a purchase of the product, or add the product toa queue (often referred to as a “shopping cart”).

Additionally or alternatively, as shown in FIG. 10B, the order interfacemay be presented by another computing device that is communicativelycoupled to the computing device used to view the visual content. Forexample, the individual may receive notifications on a mobile phonewhile viewing the visual content on an HMD or a network-connectedtelevision. The order interface may be delivered through any suitablecommunication channel (e.g., as a user interface pop-up, pushnotification, text message, or email message).

One skilled in the art will recognize that an individual may indicate aninterest in a product in many different ways. For example, theindividual may move her head toward the digital source of the scent,traverse the same location within a digital environment multiple timesin order to re-smell the scent, or perform a certain gesture (e.g., agrabbing motion) or action (e.g., pressing a mechanical button on acontrol device) that is indicative of an interest in the product.

FIG. 11 depicts a process 1100 for generating an instruction set for ascent delivery system. Digital content is initially received by arepurposing engine (step 1101). For example, the digital content may bemanually uploaded by an administrator through a GUI (e.g., GUI 514 ofFIG. 5). As another example, the digital content may be automaticallyretrieved by the repurposing engine from a content database that ishosted on a local computing device (i.e., on the same computing deviceas the repurposing engine) or a remote computing device that isaccessible over a network. The administrator may be associated with amerchant, a fragrance company, or a development company.

The repurposing engine can then identify one or more candidate instancesfor an advertisement of a product (step 1102). In some embodiments, therepurposing engine parses the digital content to automatically identifythe candidate instance(s) on behalf of the administrator. For example,the repurposing engine may apply image processing algorithms to identifycharacters, items, etc., that are embedded within the digital content.More specifically, the repurposing engine may perform image segmentation(e.g., thresholding methods such as Otsu's method, or color-basedsegmentation such as K-means clustering) on individual frames of visualcontent to isolate regions and objects of interest. In otherembodiments, the repurposing engine identifies the candidate instance(s)upon receiving user input from the administrator. For example, theadministrator may manually specify that a particular character or itemis shown in a given frame of the digital content.

A dispersal event can then be created by the repurposing engine based oncharacteristic(s) that are automatically specified by the repurposingengine or manually specified the administrator (step 1103). Thecharacteristic(s) can include, for example, which scented material(s)should be dispensed, scent duration, scent intensity, etc. Suchcharacteristic(s) are typically specified by the administrator, who maywork with a merchant, a fragrance company, or development company toidentify which scented material(s) are required to increase thedesirability of the product. Dispersal events are often offset from thecorresponding candidate instances in order to avoid the creation ofdisjointed experiences where scent production does not align with audioor video. The lead time could be measured by duration (e.g., 50milliseconds) or frame count (e.g., 5 frames). In some embodiments thelead time is fixed, while in other embodiments the lead time isvariable. For example, lead times may be increased if a series ofdispersal events (and thus a series of scents) are to be experiencedquickly in succession.

An instruction set that includes the dispersal events can be generatedfor the digital content and provided to a scent delivery system (step1104). In some embodiments the instructions are embedded within thedigital content (e.g., encoded in a video track, audio track, and/ormetadata), while in other embodiments the instructions are included in adistinct instruction set that accompanies the digital content and isseparately executable by the scent delivery system. Although the process1100 is described using visual content as an example, a similartechnique could be used to create dispersal events for non-visualcontent (e.g., audiobooks, concerts, and radio programs).

In some embodiments, an analytics module monitors the success of anadvertising campaign by observing user reactions to the scentedadvertisement (step 1105). The analytics module may be executed by therepurposing engine or a separate engine (in which case the repurposingengine and the analytics module may be hosted on different computingdevices). The analytics module may monitor the overall effectiveness ofa scent marketing campaign by analyzing different quantitative metrics,such as traffic statistics for the merchant's website, the merchant'sretail traffic, click-through rates, conversion rates, etc. Suchtechniques allow the analytics module to identify the features thatcorrespond with effective scent marketing campaigns, such as thedispersal timing, scent duration, scent intensity, scent type, scenetype, and content type. The repurposing engine can then use thisinformation to more intelligently produce scent(s) that align withcontent and lead to improvements in user engagement. For example, theanalytics module may discover that scented advertisements for a perfumeproduct are much more effective when shown before, during, or afterromance films rather than action films or comedy films.

Various machine learning algorithms and techniques could be employed bythe advertising engine, including Naïve Bayes Classifier algorithms, KMeans Clustering algorithms, Support Vector Machine algorithms, linearregression, logic regression, artificial neural networks, etc. Thesemachine learning algorithms/techniques may be chosen based onapplication (e.g., supervised or unsupervised learning) and optimizedbased on whether an administrator confirmed/denied candidate instancesthat were proposed by the repurposing engine, indicated that a certainscent created a more realistic or desirable experience, indicated that acertain scent led to more positive advertising outcomes (e.g., increasedsales or traffic), etc.

The process 1100 can be used to create direct advertisements or indirectadvertisements. However, because direct advertisements need not besubtly embedded within the digital content, it is typically much easierto identify when a direct advertisement has begun and when an individualshould be prompted to purchase the product (e.g., at the conclusion ofthe direct advertisement).

FIG. 12 depicts a process 1200 for controllably producing scents thatcorrespond with digital content experienced by a user. Morespecifically, a scent delivery system can dispense one or more scentedmaterials while the user experiences the digital content (e.g., visualcontent or non-visual content). The scent delivery system may beconnected to a computing device that is responsible for presenting thedigital content. Initially, scent circuitry of the scent delivery systemreceives an indication that a scent functionality has been enabled (step1201). The scent functionality could be enabled using a mechanicalswitch on the scent delivery system or an electronic interface presentedby the computing device. In some embodiments, the scent functionality isautomatically enabled upon determining the scent delivery system isconnected to the computing device unless the user specifies otherwise.The scent circuitry can include a power supply, memory, processor(s),and/or communication module(s) that enable the scent delivery system tocommunicate with the computing device across a wired or wirelesscommunication channel.

The scent delivery system monitors the digital content presented by thecomputing device (step 1202). The digital content could be visualcontent (e.g., cinematic videos, applications, or video games) ornon-visual content (e.g., audiobooks, concerts, and radio programs). Thescent delivery system can then generate an output signal in response todetermining a dispersal event has been detected (step 1203). Thedispersal event corresponds to a keyframe (e.g., a video frame or anaudio frame) that includes or makes reference to a particular product.Dispersal events cause the scent delivery system to produce scents thatare related to the product included in the keyframes. However, dispersalevents are often offset from the corresponding keyframes so that thedifferent sensory stimuli (e.g., smell, taste, sight, and/or hearing)are substantially synchronized. The term “keyframe” applies regardlessof whether the content is visual or non-visual. For example, bothcinematic videos and audiobooks can have keyframe(s) arranged throughouttheir runtime.

The output signal could also include other metadata about the scent thatis to be produced. For example, the output signal could specify whichpump(s) should be activated and the amount of scented material thatshould be dispensed by each pump. Output signals could also be generatedthat induce activation/deactivation of a fan (e.g., fan 208 of FIG. 2),a heating element (e.g., heating element 218 of FIG. 2), certain sensors(e.g., sensors 212 or sensors 216 of FIG. 2), or certain lightingelements (e.g., lighting elements 220 of FIG. 2). The output signals(and any corresponding metadata) could be modified based on criteriasuch as age, gender, lifestyle, usage statistics, etc. For example,scent intensity may be increased proportional to the age of the user ordecreased proportional to the current duration of usage.

Typically, the output signal is transmitted to the scent circuitry ofthe scent delivery system (step 1204). The scent circuitry processes theoutput signal in order to determine which scented material(s) should bedispensed, how much of each scented material should be dispensed, etc.The scent circuitry can then cause pump(s) to controllably dispense thescented material(s) toward the user's nose (step 1205). The scentedmaterial(s) are typically intended to produce a scent that mimics theactual scent of digital content that is included in the keyframe.However, in some instances the scent simply increases the realism ordesirability of the digital content itself. For example, anadvertisement for an athletic event or a movie may be accompanied by apopcorn scent or a caramelized sugar scent.

The amount of scented material that is released by each pump maycorrespond to the digital content being presented by the computingdevice. For example, additional scented material may be dispensed as theuser approaches the digital source of a scent. Thus, the scent intensitymay increase as the user approaches the digital source. As anotherexample, a pump may dispense less scented material if the user isconcurrently experiencing visual stimulation that is related to thescent. Said another way, smaller amounts of scented material may berequired when the user is also expected to experience significant visualstimulation and/or audible stimulation. For instance, a faint lemonaroma may be sufficient if a digital bowl of lemons is readily viewable,while a strong lemon aroma may be necessary if the user is walkingthrough a digital citrus grove whose fruit is not easily viewable.Realistic simulation of visual content requires that the strength,direction, frequency, and other features of any sensory stimuli mustgenerally correspond to what is being presented by the computing device.Accordingly, the scent circuitry may consider what each of the user'sother senses are expected to experience at a given point in time.

The scent delivery system and/or the computing device can also beconfigured to monitor the user's reaction to the scent (step 1206). Insome embodiments, the computing device and/or the scent delivery systeminclude temperature sensor(s) that monitor whether the user'stemperature has begun to rise and/or humidity sensor(s) that monitorwhether the user has begun to sweat. Both of these outcomes may beindicative of overstimulation of the user's senses. Similarly, thecomputing device and/or the scent delivery system may include motionsensor(s) that are able to determine whether the user has begun to moveher head away from the digital source of the scent or has begun to shakeher head. These actions may indicate that the scent produced by thescent delivery system is too weak or strong. A fan may circulate the airaround the user's nose or a pump may dispense a neutralizing scent toeliminate or dilute strong scents.

If the user's reaction is indicative of an interest in the product shownor referenced in the keyframe, the user may be asked whether she wishesto purchase the product (step 1107). This can occur in several differentways. In some embodiments, a notification may be presented by thecomputing device used to view the visual content. As shown in FIGS.10A-B, the notification may prompt the user to approve a purchase of theproduct, input financial information (e.g., name, address, and paymentcard details), decline a purchase of the product, or add the product toa queue (often referred to as a “shopping cart”) for further review. Inother embodiments, a notification may be presented by another computingdevice that is communicatively coupled to the computing device used toview the visual content. For example, the individual may receivenotifications on a mobile phone while viewing visual content on an HMDor a network-connected television.

Responsive to a determination that the user has asked to place an orderfor the product, the scent delivery system can inform a merchant whofacilitates completion of the transaction (step 1108). In someinstances, the financial information needed to complete the transactionmay be readily available to the merchant. For example, the merchant mayalready have such information on file from previous transactions withthe user. As another example, the scent delivery system (or anothercomputing device operated by the user) may be able to access suchinformation and provide it to the merchant.

Note, however, that scents need not always provoke order interfaces,even if those scents are intended to serve as scented advertisements.Such scenarios are another form of indirect advertising. For example,scents produced during the runtime of the digital content may correspondto a list of product(s) that are available for purchase. But the list ofproduct(s) may only be presented following to conclusion of the digitalcontent or upon request from the user.

Unless contrary to physical possibility, it is envisioned that the stepsdescribed above may be performed in various sequences and combinations.For example, in some embodiments the process for identifying candidateinstance(s) and/or specifying the characteristic(s) of each dispersalevent involves significant administrator input. In fact, targetedadvertising campaigns may require that developers work closely withmerchants, fragrance companies, and/or development companies to identifythe most appropriate scented materials, dispersal times, scentintensities, etc. Consequently, specific sets of user interfaces may beused to generate the instruction sets that are executed by the scentdelivery systems to produce scents.

Additional steps could also be included in some embodiments. Forexample, the scent delivery system may require that the user opt in oropt out of scent-based advertising campaigns. Moreover, other stimuli(e.g., gustatory or tactile, such as simulated wind) could also be usedas a component of a direct advertising campaign or an indirectadvertising campaign. For example, a direct advertising campaign for acoastal vacation resort may prompt the scent delivery system to producea salty scent and create a varied air flow that is intended to simulategusts of wind.

Emotion Manipulation

Conventional forms of digital content often use sound to manipulate anindividual's emotions. Examples of panic noises can be found in thrillerfilms and horror films, which often employ dissonant, screeching soundsthat viewers unconsciously associate with animals in distress. In fact,a study by the University of California found that human sensitivity tonon-linear alarm sounds, such as those made by groundhogs to warn aboutpredators, is employed by film composers to unsettle and unnerve. Musicis often employed with the intention that is accomplish many of the samegoals. For example, Psycho by Alfred Hitchcock includes strainingstrings and overblowing brass that mimic a panicky noise.

The relationship between sound and mood or emotional context is oftenused to guide individuals through digital content. For example, filmscores often “build up” as the intensity of a situation rises, while themusical climaxes of film scores often coincide with crucial plotsevents, both good and bad.

Scents have been shown to affect mood, work performance, and behavior ina similar manner. However, in order for a scent to elicit a response, anindividual will typically have to first learn to associate the scentwith some type of event. The process by which one event or item comes tobe linked with another because of a past experience is referred to as“associative learning.” The linked event is then able to elicit aconditioned response for the original situation. For example, a novelscent may be experienced in the context of an unconditioned stimulus(e.g., procedure in a dental clinic or hospital) that elicits anunconditioned emotional response, such as anxiety. The scent thenbecomes a conditioned stimulus for that experience and acquires theability to elicit the conditioned emotional response when encountered inthe future.

Scent delivery systems can make use of these mechanisms to manipulatemood and emotion. More specifically, scent delivery systems can producescents (or other sensory stimuli) that elicit certain emotions fromindividuals experiencing digital content.

Studies have shown that prosocial behaviors and outlook/attitude areoften enhanced in the presence of pleasant scents. Accordingly, a scentdelivery system may produce a pleasant scent as the digital contentapproaches an event having a positive outcome. The pleasant scent isintended to elicit a conditioned emotional response (e.g., happiness) byan individual, who may have smelled a prior production of the same scentor a similar scent. For example, the same pleasant scent could be usedthroughout the digital content. As another example, the pleasant scentmay mimic a scent (e.g., cotton candy or caramelized sugar) that isoften associated with positive experiences.

Improvements in mood have been linked to increases in productivity,performance, and a tendency to help others. Thus, scent production maybe particularly useful for scent delivery systems that can be used whileusers interact with the real world, such as those connected to HMDsconfigured to present augmented reality content. For example, a scentdelivery system may produce a pleasant scent (e.g., coffee orcaramelized sugar) that increases the user's real-world productivity andperformance.

Studies have also shown that unfavorable scents can lead to a negativemood that reduces prosocial behavior. Accordingly, a scent deliverysystem may produce an unfavorable scent as the digital contentapproaches an event having a negative outcome in order to buildsuspense, anxiety, fear, sadness, joy, etc. Examples of unfavorablescents include smoke, rotting flesh, and ammonia.

For example, a scent delivery system may produce a mud scent during anadvertisement for a security system. The mud scent may be accompanied byan air flow intended to simulate wind and water droplets intended tosimulate rain. The air flow may be created by the fan, while the waterdroplets may be controllably dispensed by a pump. The strength,direction, frequency, and temperature of the air flow or water dropletsmay correspond to features of the digital content presented to anindividual.

Together, these sensory stimuli may be sufficient to increase theintensity of certain emotional responses. Note, however, that scents canbe used to evoke other feelings as well. For example, food-based scentsmay be used to heighten a sensation of hunger. In some embodiments,emotions are further influenced by the dispersal of certain pheromone(s)intended to trigger specific response(s), such as alarm pheromones,signal pheromones, sex pheromones, etc.

Processing System

FIG. 13 is a block diagram illustrating an example of a processingsystem 1300 in which at least some operations described herein can beimplemented. The processing system may include one or more centralprocessing units (“processors”) 1302, main memory 1306, non-volatilememory 1310, network adapter 1312 (e.g., network interfaces), videodisplay 1318, input/output devices 1320, control device 1322 (e.g.,keyboard and pointing devices), drive unit 1324 including a storagemedium 1326, and signal generation device 1330 that are communicativelyconnected to a bus 1316. The bus 1316 is illustrated as an abstractionthat represents one or more physical buses and/or point-to-pointconnections that are connected by appropriate bridges, adapters, orcontrollers. The bus 1316, therefore, can include a system bus, aPeripheral Component Interconnect (PCI) bus or PCI-Express bus, aHyperTransport or industry standard architecture (ISA) bus, a smallcomputer system interface (SCSI) bus, a universal serial bus (USB), IIC(I2C) bus, or an Institute of Electrical and Electronics Engineers(IEEE) standard 1394 bus (also referred to as “Firewire”).

In some embodiments the processing system 1300 operates as part of anHMD (e.g., HMD 200 of FIG. 2), while in other embodiments the processingsystem 1300 is connected (wired or wirelessly) to the HMD or some othercomputing device. In a networked deployment, the processing system 1300may operate in the capacity of a server or a client machine in aclient-server network environment, or as a peer machine in apeer-to-peer network environment.

The processing system 1300 may be a server, a personal computer (PC), atablet computer, a laptop computer, a personal digital assistant (PDA),a mobile phone, a processor, a telephone, a web appliance, a networkrouter, a switch, a bridge, a console, a gaming device, a music player,or any machine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by the processing system1300.

While the main memory 1306, non-volatile memory 1310, and storage medium1326 (also called a “machine-readable medium”) are shown to be a singlemedium, the term “machine-readable medium” and “storage medium” shouldbe taken to include a single medium or multiple media (e.g., acentralized or distributed database, and/or associated caches andservers) that store one or more sets of instructions 1328. The term“machine-readable medium” and “storage medium” shall also be taken toinclude any medium that is capable of storing, encoding, or carrying aset of instructions for execution by the processing system 1300.

In general, the routines executed to implement the embodiments of thedisclosure may be implemented as part of an operating system or aspecific application, component, program, object, module or sequence ofinstructions referred to as “computer programs.” The computer programstypically comprise one or more instructions (e.g., instructions 1304,1308, 1328) set at various times in various memory and storage devicesin a computing device, and that, when read and executed by the one ormore processors 1302, cause the processing system 1300 to performoperations to execute elements involving the various aspects of thedisclosure.

Moreover, while embodiments have been described in the context of fullyfunctioning computing devices, those skilled in the art will appreciatethat the various embodiments are capable of being distributed as aprogram product in a variety of forms. The disclosure applies regardlessof the particular type of machine or computer-readable media used toactually effect the distribution.

Further examples of machine-readable storage media, machine-readablemedia, or computer-readable media include, but are not limited to,recordable-type media such as volatile and non-volatile memory devices1310, floppy and other removable disks, hard disk drives, optical disks(e.g., Compact Disk Read-Only Memory (CD ROMS), Digital Versatile Disks(DVDs)), and transmission-type media such as digital and analogcommunication links.

The network adapter 1312 enables the processing system 1300 to mediatedata in a network 1314 with an entity that is external to the processingsystem 1300 through any communication protocol supported by theprocessing system 1300 and the external entity. The network adapter 1312can include one or more of a network adaptor card, a wireless networkinterface card, a router, an access point, a wireless router, a switch,a multilayer switch, a protocol converter, a gateway, a bridge, bridgerouter, a hub, a digital media receiver, and/or a repeater.

The network adapter 1312 can include a firewall that governs and/ormanages permission to access/proxy data in a computer network, andtracks varying levels of trust between different machines and/orapplications. The firewall can be any number of modules having anycombination of hardware and/or software components able to enforce apredetermined set of access rights between a particular set of machinesand applications, machines and machines, and/or applications andapplications (e.g., to regulate the flow of traffic and resource sharingbetween these entities). The firewall may additionally manage and/orhave access to an access control list that details permissions includingthe access and operation rights of an object by an individual, amachine, and/or an application, and the circumstances under which thepermission rights stand.

The techniques introduced here can be implemented by programmablecircuitry (e.g., one or more microprocessors), software and/or firmware,special-purpose hardwired (i.e., non-programmable) circuitry, or acombination of such forms. Special-purpose circuitry can be in the formof, for example, one or more application-specific integrated circuits(ASICs), programmable logic devices (PLDs), field-programmable gatearrays (FPGAs), etc.

Remarks

The foregoing description of various embodiments of the claimed subjectmatter has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit the claimedsubject matter to the precise forms disclosed. Many modifications andvariations will be apparent to one skilled in the art. Embodiments werechosen and described in order to best describe the principles of theinvention and its practical applications, thereby enabling othersskilled in the relevant art to understand the claimed subject matter,the various embodiments, and the various modifications that are suitedto the particular uses contemplated.

Although the above Detailed Description describes certain embodimentsand the best mode contemplated, no matter how detailed the above appearsin text, the embodiments can be practiced in many ways. Details of thetechnology may vary considerably in its implementation details whilestill being encompassed by the specification. As noted above, particularterminology used when describing certain features or aspects of variousembodiments should not be taken to imply that the terminology is beingredefined herein to be restricted to any specific characteristics,features, or aspects of the invention with which that terminology isassociated. In general, the terms used in the following claims shouldnot be construed to limit the invention to the specific embodimentsdisclosed in the specification, unless those terms are explicitlydefined herein. Accordingly, the actual scope of the inventionencompasses not only the disclosed embodiments, but also all equivalentways of practicing or implementing the embodiments under the claims.

The language used in the specification has been principally selected forreadability and instructional purposes, and it may not have beenselected to delineate or circumscribe the inventive subject matter. Itis therefore intended that the scope of the invention not be limited bythis Detailed Description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of variousembodiments is intended to be illustrative, but not limiting, of thescope of the technology.

1. A stimuli delivery system that produces sensory stimuli to increasethe realism of digital content presented to an individual, the stimulidelivery system comprising: a structural frame that includes multiplefasteners for connecting the stimuli delivery system to a head-mounteddevice; a first reservoir that stores a scented liquid; a first pumpthat controllably dispenses the scented liquid from the first reservoirto produce a scent that corresponds to digital content shown by thehead-mounted device; a second reservoir that stores a flavored liquid;and a second pump that controllably dispenses the flavored liquid fromthe second reservoir to produce a taste that corresponds to the digitalcontent shown by the head-mounted device.
 2. The stimuli delivery systemof claim 1, further comprising: a fan that modulates an air flow tosimulate wind.
 3. The stimuli delivery system of claim 2, wherein astrength, a direction, a frequency, and a temperature of the air flowcorresponds to the digital content shown by the head-mounted device. 4.The stimuli delivery system of claim 2, further comprising: a heatingelement that moderates a temperature of the air flow.
 5. The stimulidelivery system of claim 1, further comprising: a processor operable toexecute instructions stored in a memory; and the memory that includesspecific instructions for producing sensory stimuli that are related tothe digital content shown by the head-mounted device, wherein executionof the specific instructions cause the processor to: parse the digitalcontent to identify a frame corresponding to a dispersal event; generatea first instruction and a second instruction responsive to determiningthat the frame has been presented to the individual; transmit the firstinstruction to the first pump, wherein the first instruction prompts thefirst pump to dispense the scented liquid from the first reservoir; andtransmit the second instruction to the second pump, wherein the secondinstruction prompts the second pump to dispense the flavored liquid fromthe second reservoir; and monitor the individual's reaction to thescent.
 6. The stimuli delivery system of claim 5, wherein the firstinstruction and the second instruction specify a particular amount, aparticular temperature, or a particular force with which the scentedliquid and the flavored liquid are to be dispensed by the first pump andthe second pump, respectively.
 7. The stimuli delivery system of claim5, wherein the frame is a video frame or an audio frame.
 8. The stimulidelivery system of claim 5, wherein the processor and the memory residewithin a remote computing device communicatively coupled to the stimulidelivery system across a network.
 9. The stimuli delivery system ofclaim 1, wherein the first reservoir is refillable only when the stimulidelivery system is separated from the head-mounted device.
 10. Thestimuli delivery system of claim 1, wherein the first reservoir ismodularly replaceable while the stimuli delivery system is connected tothe head-mounted device.
 11. A stimuli delivery system that producessensory stimuli to increase the realism of digital content presented toan individual, the stimuli delivery system comprising: a structuralframe that includes multiple fasteners for detachably connecting thestimuli delivery system to a head-mounted device; a first reservoir thatstores a scented liquid that includes a pheromone intended to induce aspecific emotional response; and a first pump that dispenses the scentedliquid from the first reservoir in a controlled manner to produce ascent that corresponds to digital content presented by the head-mounteddevice.
 12. The stimuli delivery system of claim 11, wherein the scentis produced responsive to a determination that an individual hasperformed a certain gesture detected by an optical sensor.
 13. Thestimuli delivery system of claim 12, wherein the optical sensor isaffixed to the head-mounted device.
 14. The stimuli delivery system ofclaim 11, further comprising: a second reservoir that stores water; anda second pump that dispenses water from the second reservoir in acontrolled manner to simulate rain.
 15. The stimuli delivery system ofclaim 14, wherein a strength, a direction, a frequency, and atemperature of the water dispensed by the second pump corresponds to thedigital content presented by the head-mounted device.
 16. The stimulidelivery system of claim 11, further comprising: a second reservoir thatstored a flavored liquid; and a second pump that dispenses the flavoredliquid from the second reservoir in a controlled manner to produce ataste that corresponds to the digital content presented by thehead-mounted device.
 17. The stimuli delivery system of claim 16,wherein the scented liquid and the flavored liquid are related to oneanother and dispensed substantially simultaneously.
 18. The stimulidelivery system of claim 11, further comprising: a fan that modulates anair flow in a controlled manner to simulate wind.
 19. The stimulidelivery system of claim 18, wherein a strength, a direction, afrequency, and a temperature of the air flow correspond to the digitalcontent presented by the head-mounted device.
 20. The stimuli deliverysystem of claim 11, further comprising: a second reservoir that stores amasking liquid; and a second pump that dispenses the masking liquid fromthe second reservoir in a controlled manner to neutralize the scent. 21.A method for producing sensory stimuli to increase the realism ofdigital content presented to an individual, the method comprising:receiving an indication that a sensory stimuli functionality of astimuli delivery system has been enabled, wherein the stimuli deliverysystem is communicatively coupled to a computing device that presentsdigital content to an individual; parsing the digital content presentedby the computing device to discover a frame corresponding to a dispersalevent; generating a first instruction and a second instruction inresponse to determining the frame has been presented to the individual;transmitting the first instruction to a pump of the stimuli deliverysystem, wherein reception of the first instruction causes the pump todispense a scented liquid from a reservoir to produce a scent;transmitting the second instruction to a fan of the stimuli deliverysystem, wherein reception of the second instruction causes the fan tomodulate an air flow to simulate wind; and monitoring the individual'sreaction to the scent.
 22. The method of claim 21, wherein saidmonitoring comprises: acquiring motion sensor data indicative of motionof the stimuli delivery system; parsing the motion sensor data toidentify movements of the stimuli delivery system; increasing anintensity of the scent responsive to a determination that the stimulidelivery system moved closer to a digital representation of a source ofthe scent; and decreasing the intensity of the scent responsive to adetermination that the stimuli delivery system moved further away fromthe digital representation of the source of the scent.
 23. The method ofclaim 21, further comprising: tracking a duration of usage of thestimuli delivery system; and disabling the sensory stimuli functionalityresponsive to a determination that the duration has exceeded a specifiedduration.
 24. The method of claim 21, wherein the stimuli deliverysystem is communicatively coupled to the computing device across anetwork.
 25. The method of claim 21, wherein a source of the digitalcontent provides an instruction set for producing sensory stimuli thatspecifies the dispersal event.
 26. The method of claim 25, wherein theinstruction set is encoded in a video track, an audio track, or metadataof the digital content.
 27. The method of claim 25, wherein theinstruction set accompanies the digital content and is separatelyexecutable by the stimuli delivery system.
 28. A method for producingsensory stimuli to increase the realism of digital content presented toan individual, the method comprising: receiving an indication that asensory stimuli functionality of a stimuli delivery system has beenenabled, wherein the stimuli delivery system is communicatively coupledto a computing device that presents digital content to a user; parsingthe digital content presented by the computing device to discover aframe corresponding to a dispersal event; generating a first instructionresponsive to a determination that the frame has been presented to theuser; transmitting the first instruction to a pump of the stimulidelivery system, wherein reception of the first instruction causes thepump to dispense a scented liquid from a reservoir to produce a scent;parsing sensor data that is indicative of user interactions with thedigital content presented by the computing device; generating a secondinstruction responsive to a determination that the user has performed acertain action; and transmitting the second instruction to the pump ofthe stimuli delivery system, wherein reception of the second instructioncauses the pump to increase an intensity of the scent by dispensingadditional scented liquid from the reservoir.
 29. The method of claim28, wherein the sensor data is generated by a motion sensor or an audiosensor.
 30. The method of claim 28, wherein the certain action includesan interaction with a digital representation of a source of the scent, atraversal of a digital environment in proximity to the digitalrepresentation of the source of the scent, or both.